Using Predictive Representations to Improve Generalization in Reinforcement Learning
نویسندگان
چکیده
The predictive representations hypothesis holds that particularly good generalization will result from representing the state of the world in terms of predictions about possible future experience. This hypothesis has been a central motivation behind recent research in, for example, PSRs and TD networks. In this paper we present the first explicit investigation of this hypothesis. We show in a reinforcement-learning example (a grid-world navigation task) that a predictive representation in tabular form can learn much faster than both the tabular explicit-state representation and a tabular history-based method.
منابع مشابه
Relational Representations in Reinforcement Learning: Review and Open Problems
This paper is about representation in RL. We discuss some of the concepts in representation and generalization in reinforcement learning and argue for higher-order representations, instead of the commonly used propositional representations. The paper contains a small review of current reinforcement learning systems using higher-order representations, followed by a brief discussion. The paper en...
متن کاملLearning to Generalize through Predictive Representations: A Computational Model of Mediated Conditioning
Learning when and how to generalize knowledge from past experience to novel circumstances is a challenging problem many agents face. In animals, this generalization can be caused by mediated conditioning—when two stimuli gain a relationship through the mediation of a third stimulus. For example, in sensory preconditioning, if a light is always followed by a tone, and that tone is later paired w...
متن کاملLearning Methods for Sequential Decision Making with Imperfect Representations
Sequential decision making from experience, or reinforcement learning (RL), is a paradigm that is well-suited for agents seeking to optimize longterm gain as they carry out sensing, decision, and action in an unknown environment. RL tasks are commonly formulated as Markov Decision Problems (MDPs). Learning in finite MDPs enjoys several desirable properties, such as convergence, sample-efficienc...
متن کاملState Representation Learning in Robotics: Using Prior Knowledge about Physical Interaction
State representations critically affect the effectiveness of learning in robots. In this paper, we propose a roboticsspecific approach to learning such state representations. Robots accomplish tasks by interacting with the physical world. Physics in turn imposes structure on both the changes in the world and on the way robots can effect these changes. Using prior knowledge about interacting wit...
متن کاملXcs Classiier System Reliably Evolves Accurate, Complete, and Minimal Representations for Boolean Functions. Technical Report Version
This paper extends the work presented in (Kovacs, 1996) on evolving optimal solutions to boolean reinforcement learning problems using Wilson's recent XCS classiier system. XCS forms complete mappings of the payoo environment in the reinforcement learning tradition thanks to its accuracy based tness, which, according to Wilson's Generalization Hypothesis, also gives XCS a tendency towards accur...
متن کامل